Sound ”Gesturefication”, sound/gesture-mapping for the interaction with recorded sounds

With ”Gesturefication” we designate the creation of a mapping between gestural control and sound synthesis parameters based on the recording and analysis of a sound and related gestures. These gestures may have actually produced the sound or simply describe the sound from a particular point of view such as instrument performance, music conducting or active and embodied music listening. Our working hypothesis is that mappings can be derived from the coherence of data sets describing the gestures on one hand and the sound on the other. In the ideal case, these mappings have to take into account their temporal evolution as well as the recurrence of patterns in both processes. This paper describes a first approach using an adaptive model for the mapping between instantaneous motion and audio features.